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Abstract 

The undergraduate seienee programme was launehed at the Indira Gandhi National Open 
University (IGNOU) in 1991-92 with an enrolment of 1,210 students. The programme was well 
reeeived, and enrolments inereased over the years. However, the sueeess rates have not kept paee 
with enrolment. 

In this paper, the authors report the results of an evaluation of the undergraduate physies 
programme at IGNOU. The evaluation, the first of its type for this programme, adapted the major 
tenets of the CIPP model. The findings are based on the responses from a randomly ehosen 
sample of 509 learners aeross India. The methods employed for the study inelude reeords, 
doeument, and database analysis, surveys, and ease studies. 

Although the University has enhaneed aeeess to higher seienee edueation, the attrition rate is high 
(73%), and the sueeess rate is low. The authors reeommend that the University review and 
reorient its strategies for providing good quality, leamer-eentred higher edueation in seienee 
subjeets. The programme should address the eoneerns of the learners about the effeetiveness of 
the student support systems, the diffieulty level, and the learner-friendliness of study materials 
with the goal of aehieving long-term sustainability while maintaining parity with the eonventional 
system. The need for improving the presentation of the eourses and simplifying the mathematieal 
details is emphasised. 
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Undergraduate Physics Programme at Indira Gandhi National Open 

University 

The undergraduate science programme aimed at providing higher science education through the 
distance mode at the national level was launched by the Indira Gandhi National Open University 
(IGNOU) in December 1991 with an enrolment of 1,210 students. The annual enrolment reached 
9,973 in 2003 before stabilising at about 6,000 in 2006. The programme design is based on a 
flexible credit system and is characterised by a three-tier course structure: compulsory foundation 
courses (24 credits), elective courses to provide for core areas of study in a discipline (56 to 64 
credits), and application-oriented courses (8 to 16 credits) to provide rudimentary skills that help 
to improve the employment prospects of the graduates. Learners who earn 48 credits in elective 
courses in any one discipline are awarded the B.Sc. degree in that subject (Vijayshri et ah, 1998, 
p. 109). 

The programme is flexible and open as far as the course options, the place, and the pace of 
learning are concerned. It is offered in five science disciplines (physics, chemistry, mathematics, 
life sciences or botany, and zoology), and students must opt for a minimum of 25% of total 
elective credits earned in disciplines with experimental components from laboratory courses. The 
instructional methodology comprises multiple-media self-instructional packages, with print as the 
mainstay. The assessment has two components: continuous and term-end. IGNOU provides 
support services to its B.Sc. students through a total of 1,653 study centres (SCs), out of which 
150 (SCs) are activated for the B.Sc. programme. (SCs have been activated in reputed institutions 
[colleges and university departments] of higher learning to ensure that appropriate academic and 
other support is provided to the students.) The laboratory training and academic counselling for 
courses in the B.Sc. programme are carried out at all 150 SCs. 

The programme has been well received as demonstrated by the increasing enrolments over the 
years, and it has been welcomed by those who missed the opportunity for higher science 
education. However, over the years, students, academic counsellors, and physics experts have 
expressed concerns to two of the authors about the quality of the curriculum transaction, 
including the laboratory, the effectiveness of the student support system, and the performance and 
satisfaction of the learners. Another motivation for the study was the decision taken by the Board 
of Management of IGNOU in 2002 to revise all programmes so that the shelf life of a programme 
is not more than seven years. To address these issues, it was considered essential to undertake a 
systematic evaluation of the physics programme, which is offered as a part of the bachelor’s 
degree programme in science at IGNOU. In the present study, the major tenets of the CIPP model 
have been adapted. 


Programme Evaluation 

In distance education, programme evaluation encompasses the evaluation of programme 
objectives, course content, instructional design, support services, assessment practices, student 
achievements, and use/impact of technology (IGNOU, 2006) so as to improve the quality of 
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teaching-learning, to enhance the relevance of the programme, to assess how the programme is 
perceived by its stakeholders and meets their expectations, and to provide regular feedback on the 
factors that affect outcomes. There exists substantial literature on the theory and methods of 
programme evaluation (Kellaghan & Stufflebeam, 2003, p. 1-79), which delineates the objectives, 
approaches, and methods for evaluating a programme (Mishra, 2008, p. 34-46). In the present 
study, we have adapted the major tenets of the CIPP model (Stufflebeam, 1983, p. 117). The 
CIPP evaluation model promotes the view that the purpose of evaluation is programme 
improvement. The process involves four stages of evaluation: context, input, process, and 
product. 

A few studies on evaluation of programmes offered in the distance mode are available in the 
public domain. The PhD programme in Education at the University of the Philippines (UP) Open 
University has been evaluated using the CIPP model (Quimbo, 2002, p. 196). The study revealed 
that improvement in significant input and process variables made a positive impact on learner 
performance. The CIPP model was also used to monitor and develop an evaluation framework for 
technology-based community learning centres in India (Calder & Patel, 2002, p. 214). 

Kapoor (2004) evaluated the Mental Health Nursing Care course - a competency-based course - 
offered in the B.Sc. Nursing programme at IGNOU and used a model akin to the CIPP evaluation 
model. The study revealed that the self-learning materials (SLMs) and laboratory experience for 
this course were interesting and useful but access to electronic media facilities at the study centres 
was poor, even though tele-conferencing had been made mandatory. 

In a recent study, Fozdar et al. (2006) reported a variety of factors (personal, academic, and 
financial) that were responsible for the dropout of B.Sc. students at IGNOU. The study reported 
in this paper is comprehensive and the first of its kind on the evaluation of the B.Sc. Physics 
programme offered by IGNOU. Our evaluation model is essentially summative in intent and 
structure (Panda, 1991, p. 168). Figure 1 shows the conceptual framework developed for this 
study based on the theoretical and empirical evidence. Although many aspects of programme 
evaluation have been included in Figure 1, in the study reported here we have included only the 
objective set provided below. 

Research Objectives 

The present study is limited to evaluating the programme for the following: 

• access, equity, and relevance to learner needs and expectations; 

• success rates of enrolled sampled students; and 

• evaluation of the quality, presentation, and usefulness of self-instructional study materials. 

Findings pertaining to programme evaluation of other aspects listed in Figure 1, for example the 
quality of the student support services and the efficacy of the administrative and managerial 
system, will be reported in forthcoming papers. 
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Context evaluation 

• Relevance and objectives of the programme 

• Learner readiness for the programme 

• Parity with contemporary programmes 

Input evaluation 

Quality of study materials in terms of their 
adequacy, conceptual correctness, and level 
Presentation, i.e., coherence, sequencing, language, 
printing, graphs, data, style 
Quality of student support services 
Pedagogical effectiveness: learner-friendliness, 
comprehensiveness, difficulty level 
Use of technology - extent and diversity 
Administrative system and support for delivery of 
materials, timely declaration of results, conduct of 
counselling sessions, library facilities promote self- 
learning 

Quality of curriculum transaction in counselling, 
practical sessions, and assignment evaluation 
Workload and other pedagogical issues 
Material management 
Learner satisfaction 

Relevance, innovations, and cost-effectiveness 
Assessment 


.Product evaluation 

Undergraduate physics 
programme 



Figure 1. Conceptual framework for evaluation of undergraduate physics programme of IGNOU. 


The research design formulated vis-a-vis the objectives of programme evaluation encompasses 
studying the feedback of primary sources (learners, counsellors, programme designers and 
developers) on various aspects of the programme, collection of institutional data, and responses 
of various stakeholders. This study was initially formulated and designed in 2005 following the 
decision of the University in 2002 that every programme launched more than a decade ago should 
be reviewed. The period of the study was taken to be from the year 2001 to 2005 because the 
programme had stabilised by then, and the first batch of learners had completed the maximum 
period (eight years then but six years now) for completion of the programme. 


Research Methods 


The following methods were adopted to carry out this study: 

• Records, document, and database analysis'. Institutional data, documents, and records 
were used to source information on the philosophy, guiding principles and practices for 
design, development, and offer of the B.Sc. programme and data on various aspects of the 
programme, namely learner enrolment, learner characteristics (employment status, social 
status, gender, etc.), and success rates. 

• Surveys'. Questionnaires, interviews, observation, and focus group discussion were used 
to probe the variables and trends emerging from the analysis of institutional records and 
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personal interactions of the researchers with various stakeholders. Feedback was obtained 
through questionnaires (structured, semi-structured, and open-ended) and interviews 
using multiple channels of communication on aspects such as course curriculum, course 
content, learning experiences, performance, and outcomes. 

• Case studies'. Case studies of select B.Sc. Physics students were undertaken to bring out 
the experiences of the learners and to develop a deeper understanding of the issues 
involved. 


Population and Sample 

The population for the study initiated in 2005 was distributed over the 146 study centres that were 
activated by IGNOU for its B.Sc. programme. The prime consideration of the researchers was to 
select a sample representative of the system and to ensure that the selected centres represented an 
all-India picture. A survey of the institutional enrolment data of learners for the years 200 1 to 
2005 revealed that 45-50% of learners enrolled in the B.Sc. programme from the Northern region, 
25-30% from the Eastern region, 15-20% from the Southern region, and 5-10% from the Western 
region. 

In this study, questionnaires were administered to about 800 learners and responses were received 
from 509 students (64%). (One of the researchers personally visited 16 study centres in different 
parts of the country.) The maximum number of sampled students was from academic session 
2005 (224, 43%), followed by academic session 2004 (125, 25%) and academic session 2003 
(115, 23%). Only 45 learners in the sample (9%) were enrolled in 2001 and 2002. This indicates 
that most respondents were in the first, second, and third year of the programme and had the most 
recent experiences of the programme. Therefore, this sample can be taken as authentic in so far as 
providing feedback about various aspects of the programme is concerned. The sample size also 
meets the requirement of 99% confidence level and 5% confidence interval for the average 
number of students enrolling during the period 1997-2005. 

The learner population was sampled at different stages of the study. The objectives of the study 
suggested that a sample needed to be determined for those who wished to major in physics and 
pursue higher education and for those who were interested in upward mobility of their career in 
science laboratories (school/college/university/R&D). A sub-set of 199 learners (39%) who were 
pursuing the B.Sc. in Physics, out of the total sample of 509, was identified, and information on 
major trends emerging from the analysis of responses was verified in tele-interviews and face-to- 
face interactions. The institutional data was obtained from the Student Registration & Evaluation 
Division (SR&ED) of IGNOU. 


Enrolment Pattern in Physics Courses 

The enrolment data in various physics courses for the period 2000 to 2006 and its comparison 
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with the total enrolment data (soureed from institutional reeords) revealed that 

• on an average, about 42% of the total number of learners enrol in the three 1 level eourses 
spread over eight eredits; 

• enrolment deereases as the level of the eourses inereases; 37% of learners enrol in five 
level eourses spread over 16 eredits and about 21% of students enrol for level eourses 
worth 24 eredits, out of 32 eredits worth of eourses available; and 

• enrolment in speeialised eourses is low. 

This behaviour suggests that 3’^'* level physies eourses are usually seleeted by those learners who 

are interested in pursuing a related master’s degree or a eareer in physies. 


Analysis of Institutional Data for Success Rates of Learners 


The sueeess rates, retention rates, and performanees of learners are important indieators of the 
aeeeptability and sustainability of a programme. To diseover these for learners opting for physies 
eleetives (listed in Table 1) or opting to pursue a major, the institutional data was serutinised. It 
revealed the following: 

• 63,478 learners enrolled in the fr* year of the B.Se. programme up to the 2007 admission 
year. Of these, 28,935 learners (46%) enrolled in the 2"‘* year and 17,289 (27%) enrolled in 
the 3"^*^ year. Thus, the retention rate in the B.Se. programme is about 27% and the attrition 
rate is about 73%. This indieates that IGNOU has ereated opportunities for learning higher 
seienee at a distanee, but there is a need to devise effeetive strategies to enhanee the 
programme eompletion rate. 

• Until the 19* eonvoeation held in 2008, only 3,000 learners had earned the B.Se. degree, 
whieh means that only about 5% of the total enrolled learners and 17% of those who enrolled 
in the 3'^‘^ year were able to eomplete the programme sueeessfiilly. Of these, 1,443 learners 
had earned the general B.Se. and 1,557 had earned a major in speeifie seienee diseiplines 
(botany, ehemistry, life seienees, mathematies, physies, and zoology); 346 earned the B.Se. in 
Physies. Obviously, this dispels the fear that seienee eannot be learnt at a distanee. But the 
low sueeess rate of learners has implieations for the University as far as reviewing its 
strategies for programme design, development, and implementation are eoneemed. 

• Of the sueeessful physies learners, 82% took 3 to 6 years, 9% spent the minimum period of 3 
years, about 33% took 4 years, and 28% spent 5 years. This means that learners begin to lose 
motivation after about 7 to 8 years. 

In Table 1, we have listed year- wise pass pereentage in physies eleetives PHE-01 to PHE-14 
(offered until 2005) through the years 2000 to 2005 based on the institutional reeords. The 
numbers in braekets indieate the eredit weight of eaeh eourse. The eorresponding values of 


111 


Evaluation of the Undergraduate Physics Programme at Indira Gandhi National Open University: A Case Study 

Mishra, Vijayshri, and Garg 


average pass pereentage, standard deviations, and t-values are also given. We diseovered the 

following: 

• The sueeess rates of learners opting for physies eleetives vary from about 28% to 53% in 
theory eourses and 65% to 98% for praetieal eourses. 

• Learner performanee at all three levels shows statistieally signifreant differenees in laboratory 
eourses vis-a-vis theory eourses, being far better in the former, possibly due to F2F 
interaetion with teaehers in the laboratories. The differenee in mean pass pereentages is 
statistieally signifreant in the laboratory eourses at different levels, with student performanee 
improving as the level of the eourse gets higher. This finding eorrelates well with the 
enrolment pattern, whieh shows that the higher level physies eourses are opted for by students 
pursuing a physies major degree. 

• The differenee in mean pass pereentages in theory eourses is statistieally not signifieant at 
any level, exeept PFIE-06 vis-a-vis other theory eourses at the seeond level and PFIE-1 1 vis-a- 
vis other theory eourses at the third level. It is elear that learners pereeived these eourses as 
diffieult, although no radieal ehanges in the eontents were suggested either by the experts or 
by the eounsellors of these eourses. This refleets a gap between the pereeptions of the 
learners and of the providers, and there is a need for improvement in the eourse presentation, 
in the review of term-end examination (TEE) question papers, and in the quality of aeademie 
support. 

• There is no statistieally signifieant differenee in the performanee of learners in pre-requisite 
eourses at any level (PHE-10 and PHE-07; PHE-14 and PHE-04/PHE-05; PHE02 and PHE- 
09 
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Table 1 

Year- Wise Pass Percentage in Physics Electives in the Period 2000-05 


Course 

PHF 

PHF 

PHF 

PHF- 

PHF- 

PHF- 

PHF- 

PHF- 

PHF- 

PHF- 

PHF- 

PHF- 

PHF- 

PHF 

Year 

-01 

-02 

-03L 

04 

05 

06 

07 

08L 

09 

10 

11 

12L 

13 

-14 (4) 


(2) 

(2) 

( 4 ) 

(2) 

(2) 

( 4 ) 

( 4 ) 

( 4 ) 

( 4 ) 

( 4 ) 

( 4 ) 

( 4 ) 

( 4 ) 


2000 

34 

24 

- 

- 

51 

19 

36 

- 

13 

50 

10 

- 

- 

- 

2001 

32 

27 

53 

47 

32 

14 

49 

64 

26 

37 

12 

- 

- 

- 

2002 

55 

45 

60 

48 

50 

26 

57 

59 

45 

61 

44 

100 

- 

66 

2003 

26 

26 

47 

25 

48 

24 

54 

66 

37 

49 

16 

94.2 

30 

57 

2004 

18 

22 

69 

43 

44 

21 

45 

72 

39 

30 

20 

98.3 

36 

46 

2005 

29.2 

25.2 

98.6 

48.8 

33 

34 

29.2 

98 

48 

38.2 

53 

98.4 

59.3 

43 

Mean 

32.4 

28.2 

65.5 

42.4 

43 

23 

45.0 

71.8 

34.7 

44.2 

25.8 

97.7 

41.8 

53 

Std Dev 

12.4 

8.40 

20.23 

9.96 

8.49 

6.81 

10.69 

15.37 

13.06 

11.21 

18.12 

2.47 

15.48 

10.55 

t-value 

2.571 

2.571 

2.776 

2.776 

2.571 

2.571 

2.571 

2.776 

2.571 

2.571 

2.571 

3.182 

4.303 

3.182 


PHE-01: Elementary Mechanics; PHE-02: Oscillations and Waves; PHE-03L, 8L and 12L Physics Laboratory-I, II 
and III; PHE-04, PHE-05 and PHE-14: Mathematical Methods in Physics - 1, II and III; PHE-06: Thermodynamics 
and Statistical Mechanics; PHE-07: Electric and Magnetic Phenomena; PHE-09: Optics; PHE-10: Electronics and 
Electrical Circuits; PHE-1 1: Modern Physics; PHE-13: Physics of Solid 

The one-way ANOVA test, whieh ean be thought of as an extension of unpaired student t-test, 
was earried out for analysis of varianee for two sets of theory and lab eourses. The ealeulated p 
and F values for the lab eourses are respeetively 0.02 and 5.31. The value of F is greater than the 
theoretieal value of 3.98 indieating that the average pass pereentages in lab eourses are 
statistieally signifieant. But in the ease of the Mathematieal Methods in Physies eourses, p and F 
values are respeetively 0.223 and 1.71 (< 3.89), indieating that in this ease average pass 
pereentages are statistieally not signifieant. 


Analysis of the Feedback Data and Major Findings 

We now present the findings of our study in respeet of the profile of respondents, learners’ 
feedbaek on eourse materials, and their validation by eounsellors and programme developers. The 
results have been presented ehiefly through eomputation of pereentages. 

Respondents’ Profile 

The basie eharaeteristies of 509 sampled learners revealed that the sampled population was 
predominantly urban (89%) and belonged to the non-Seheduled Caste/Seheduled Tribe eategory. 
(These have long been marginalised seetions of Indian soeiety, deprived eeonomieally, soeially, 
and edueationally.) A signifieant majority (84%) was over 21 years, male (70%), and lived far 
(25-50 km) from the study eentres (68%); a learner might take 2-3 hours eommuting eaeh way, 
depending on the geographieal loeation and availability of transport. The population of employed 
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and not employed learners was nearly equal, and English as the medium of instruetion was not 
pereeived as a barrier. 

To explore the reasons for the low sueeess rates, learner responses on the time they devoted to 
their studies and on their study habits were examined. The findings are given below. 

Study Time and Study Habits 

In view of their soeietal, familial, and personal eommitments, distanee learners invariably 
experienee aeute shortages of time. And a frequently asked question eoneems the number of 
hours they must study everyday or in a week to sueeessfully eomplete a programme in the 
minimum time without eompromising the quality of learning. This has implieations for eourse 
design, development, and student workload. Our analysis showed that 51% of learners (252) put 
in more than 6 hours per week; however, half of the students spend, on an average, less than one 
hour per day. 

Unfortunately, no doeument advises learners that spending 2-3 hours on an average everyday is 
neeessary for sueeessfiil eompletion of the programme in the minimum speeified duration of three 
years. Although in all physies eleetives, the learner is advised about the estimated time s/he is 
expeeted to spend on eaeh unit, bloek, and eourse, it is suggested that the programme 
developers/faeulty members should also advise learners about the average time they must spend 
everyday to sueeessfully eomplete the programme in minimum time. It is therefore desirable to 
inelude a detailed guide in the learners’ programme materials on ‘how to study on their own at a 
distanee’ and reinforee this adviee in eounselling/orientation sessions through eleetronie media as 
well as through F2F eounselling at the study eentres. 

In response to the speeifie question on how they study the print materials (e.g., a new bloek/unit), 
361 learners (74%) reported that they began to read the unit from the beginning, and only 24 
learners (5%) read the bloek introduetion and the study guide to the bloek. Fifty-five learners 
(11%) began from the strueture, 22 (4.5%) read the eourse objeetives, and 14 (3%) gave 
eomments sueh as laek of time, eonsulting other referenee books, highlighting main points. This 
shows that the majority of sampled learners were serious about their studies, and they should be 
advised about the importanee of the objeetives in every unit as the yardstiek against whieh they 
are tested and against whieh they should measure their learning aehievements. 

Learners’ Feedback on Course Materials 
Quality of print materials 

Printed self-instruetional materials form the baekbone of instruetion in IGNOU. Other media 
(A/V and web-based) are used as supplements only. Therefore, evaluating various aspeets related 
to teaehing-leaming through the eourse materials was a major eoneem of the researehers. The 
data analysis revealed that 99% of responding learners appreeiated the eourses and the majority of 
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learners rated the eourses as good (293, 66%), although the teaeher only eommunieated with them 
sometimes (403, 82%) (in text), and the study material was partially self-instructional (316, 65%). 
Considering the breadth of eourse material, three out of four learners (374, 75%) opined that the 
material was adequate; whereas, 64 learners (13%) thought otherwise. In addition, 60 students 
(12%) found the material lengthy. But learners who did not find the eourses adequate gave no 
suggestions on the topies to be ineluded or deleted. 

A majority of respondents (413 out of 496) reportedly needed help to learn the eourses. The 
major reasons eited were high content density, difficulty in following the mathematical treatment, 
paucity of good diagrams, and lack of sufficient number of examples and exercises for practice. 
From these findings one may eonelude that the learning materials do not fully substitute for the 
teaeher, and it is prudent to revise them by addressing the eoneems of learners. 

Relevance of course content. 

A signifieant majority of respondents (400, 83%) pereeived the eourse eontent as relevant, while 
16 learners (3%) opined that the materials were not relevant, and 65 learners (13%) were 
undeeided. 

Presentation of print materials. 

The analysis of the data on sequeneing and presentation of printed study materials revealed the 
following: 

• 87% of respondents (381 out of 438) preferred a statement of the objeetives at the beginning; 
whereas, 12% of learners (54) opined that they be shifted to the end of the unit. However, no 
argument was given in support of either ehoiee. 

• 91% of respondents (443 out of 489) pereeived the summary as useful. 

• 82% of learners (405 out of 492) expressed satisfaetion on language eomprehensibility. 
However, 67% of learners (333) would like more illustrations, whieh has important 
pedagogieal implieations as a figure ean effeetively eommunieate intrieate 
pattems/behaviours. 

• 84% of responding learners (413 out of 492) found the learning materials to be of a high 
aeademie standard. 

Formative self-assessment. 

The in-text self-assessment questions (SAQs) and terminal exereises (TQs) in the eourse 
materials are intended to elarify eoneepts further and to provide opportunities for self-leamers to 
assess their understanding of the eontent. In our sampled population, 140 learners did not respond 
to the question on whether they attempted the self-eheek questions/exereises/aetivities ineluded in 
their study materials. Of 369 learners, 347 (94%) reported that they attempted SAQs and TQs. 
This suggests that about 30% of learners in the sample were not self-evaluating. A large number 
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of students (415, 86%), in spite of adviee to the eontrary, eheeked the SAQ answers sometimes 
before attempting them. However, SAQs were found most helpful in elarifying diffieult eoneepts 
by 484 out of 496 respondents (97.6%) (Table 2). And only a few learners re-wrote answers if 
they were not satisfied with their responses. 

Table 2 

Usefulness of In-Text Questions to Understand Content 


Response 

Respondents* 

Very helpful 

144 (29) 

Reasonably 

helpful 

340 (69) 

Not helpful at all 

12(2) 

Total 

496 


(100) 


*No response =13 


Table 3 

Ways Learners Found Self-Assessment Questions Helpful 


Response 

Respondents* 

Help to elarify eoneepts 

146 (43) 

Help solve assignments and paee learning 

55(16) 

Provide useful feedbaek on the understanding of subjeet 

40 (12) 

Help prepare for TEE well 

101 (29) 

Total 

342 (100) 


*No response = 1 


Although few learners responded to the question about their preferred type of SAQ (objeetive 
type: multiple ehoiee, fill in the blanks, matehing, true/false; short answer type: sequeneing, 
numerieal; and long answer type: derivation, applieation), from personal interaetions with the 
students, the researehers have formed the view that multiple ehoiee questions in the objeetive 
type, numerieal in the short answer type, and applieations in the long answer type should help 
learners to gain mastery over the subjeet. 

In Table 3, we have listed the responses to the open-ended question about the ways in whieh 
learners found the SAQs helpful. In a nutshell, SAQs and TQs play an important role in 
faeilitating learning. The implieations of these findings for eourse developers are that the number 
of in-text questions should be inereased, and they should be diverse; the problems should 
eorrelate learning with real-life situations; and the questions should be sequeneed in order of 
diffieulty and of learning objeetives. 
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Difficulty level of learning materials. 

The learners were asked to rate the diffieulty level of eoncepts, mathematieal derivations, 
examples and exereises, assignments, ete. on a four-point seale. 

Table 4 

Learner Rating of Difficulty Level of Materials 


Aspects 

Easy 

Slightly 

difficult 

Difficult 

Very 

difficult 

Total 

Explanation of 
concepts 

157 (32) 

250 (51) 

78(16) 

4(1) 

489 (100) 

Correlating knowledge 
with surroundings 

121 (27) 

252 (55) 

77(17) 

3(1) 

452 (100) 

Mathematical 

derivations 

125 (26) 

162 (34) 

169 (35) 

24 (5) 

480 (100) 

Examples and self- 
check exercises (SAQs 
-TQs) 

300 (64) 

130 (28) 

34 (7) 

6(1) 

470 (100) 

Assignments 

99 (20) 

166 (34) 

184 (38) 

37(8) 

486 (100) 

Presentation of 
concepts 

196 (42) 

205 (44) 

55 (12) 

10(2) 

466 (100) 


The data given in Table 4 suggests that more than 80% of responding learners eonsider the 
following as easy or slightly diffieult: explanations/presentation of eoneepts, eorrelation of 
knowledge with immediate surroundings, quality of examples, and self-eheek questions (SAQs 
and TQs). However, 40-45% of responding learners find mathematieal derivations and 
assignments diffieult or very diffieult. This eould be the major reason for the laek of satisfaetion 
of learners with materials. It would be worthwhile to provide eomplete derivations without 
skipping in-between steps. 

Learner feedbaek on eourse materials obtained through questionnaires and interviews informs us 
that the level of materials should be moderated, and their presentation should be in tune with 
learner eapabilities, expeetations, and interests. It is prudent to improve the presentation by 
reiterating important definitions and highlighting them in boxes, for example, and by 
ineorporating more illustrations, bar eharts, tables, and solved and unsolved examples. 
Mathematieal steps in the text should be explained in detail and continuous assessment through 
assignments should be in consonance with course contents. These findings may serve as general 
guidelines for developing revised versions of the courses. In the ultimate analysis, the revised 
version of each course should address course-specific learner feedback. 

Learners in IGNOU come from varying backgrounds and constitute a heterogeneous group in 
competences, skills, and learning styles. But it seems that programme designers and developers 
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have pitched the physics courses at a level higher than that expected by an average learner due to 
the over-riding concern for parity with conventional universities. This means that all physics 
courses should be moderated by simplifying language, by including more examples, diagrams, 
and pictures, and by solving intermediate mathematical steps. Modifications may also include 
regional requirements, practices, and variations. These findings have significant implications for 
learner performance and retention, programme revision, implementation, and sustainability. 
These provide crucial input for distance educators in their efforts to promote effective and active 
self-learning and to bridge the gap between the perceptions of programme designers/developers 
and learners. 


Analysis of Responses of Learners Pursuing the B.Sc. Degree in Physics 

In the sample of 509 learners, 199 learners were pursuing the B.Sc. Physics degree. While this 
sub-group, in general, exhibited behaviour very similar to that of the sampled population, a few 
distinguishing features emerged. These are outlined below: 

• The percentage of majoring respondents who intended to pursue research as a career rose to 
4 1 % (from 2 1 % in the total sample). 

• A greater proportion of majoring respondents, who attended counselling sessions every week, 
perceived mathematical derivations, examples, and exercises as easy. 

• While 77 learners needed guidance in only the theory courses, 106 learners needed guidance 
in both theory and laboratory courses. 

The credit completion status of 193 responding major learners at the time of the study is given in 
Table 5. For comparison, we have also included the status of 295 non-majoring and 488 of the 
sampled population. A closer examination of the status of 42 majoring learners who had 
completed 76-100% of credits revealed that 30 learners had completed the B.Sc. in Physics and 
12 were about to complete all courses. Moreover, the percentage of learners earning more than 
25% but less than 50% of the credits was more for the majoring group. So we can conclude that 
the completion rate of the learners intending to earn the B.Sc. in Physics within the stipulated 
time period of 3-6 years is better than that of non-majoring students. 

Table 5 

Credit Completion Status of Different Groups of Learners 


Group 
% credits 
completed 

Majoring sub- 
group* 

W=193 

Non-majoring 
sub-group 
A =295 

Entire sample* 

<25 

59 (30.6) 

89 (30) 

148 (30) 

26-50 

61 (31.6) 

65 (22) 

126 (26) 

51-75 

31 (16) 

39(13) 

70 (14) 


144 (30) 


76- 100 42 (21.8) 102 (35) 

* 1 dropped and 5 changed electives; * 5 dropped and 20 changed electives 
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Laboratory Instruction 

In imparting instruction in science at a distance, practical training is construed as the major 
impediment by conventional peers. To provide an interactive, creative, and stimulating 
experience, the physics curriculum in IGNOU places greater emphasis on day-to-day work in the 
lab, which is accorded 70% of the final grade. The detailed findings have been presented 
elsewhere (Mishra et al., 2008). It may suffice to say that contrary to the expectations of the 
purists among academia, learners preferred to earn the maximum possible credits in practical/lab- 
based courses. But the basic considerations of learners seem to be full-time F2F guidance, a high 
success rate, and better grades. About 90% of distance learners opined that lab sessions were well 
planned and structured, lively and stimulating, and helped them to learn a lot of good physics; 
further, they looked upon practical sessions as an enjoyable experience. But some of these 
findings were not confirmed by the faculty. Moreover, there seems to be no correlation in the 
grades of learners in practical and theory courses (Khare et al., 2004). 

Case Studies 

To validate the patterns and characteristics emerging from the analysis of the responses of the 
majoring cohort, a research tool was administered on five learners in the form of case studies. 
Their written responses received by email were followed up by telephone interviews. Among the 
reasons they gave for joining IGNOU were a lack of time for full-time studies and the opportunity 
to pursue other courses simultaneously. The researchers encouraged them to give their frank 
assessment of the quality of course materials and of how the B.Sc. Physics degree helped them to 
shape their careers. They were also asked to suggest ways to improve systemic efficiency and to 
promote learner performance. 

The programme-specific strengths identified by the respondents included flexibility in the choice 
of subjects and in the pace of learning, quality of learning materials, and equal opportunity to 
study science at the tertiary level. But limited use of ICTs in the teaching of physics and non- 
inclusion of project work and computer courses in the curriculum were identified as the major 
deficiencies. 


Feedback from Counsellors and Programme Developers 

Fifty-four counsellors were shortlisted to provide feedback: 34 sent their responses by post (group 
I), and 20 were interviewed in person (group II). The majority of respondents were male (45) and 
possessed a PhD degree; some of them were university professors/former heads of 
department/principals of colleges. All of them had good exposure to the courses being counselled, 
and many were counselling more than one course. The spread of their institutions ranged from 
leading institutions in cosmopolitan cities to lesser known colleges in towns. We can safely 
assume that their responses represent a rich pool of diverse practices, comparative positioning of 
courses, and learning environment/habits/styles of IGNOU learners enrolled in this programme. 
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To know how counsellors perceived physics materials, specific questions were raised about 
curriculum design, content presentation, difficulty level, and possible improvements. Twenty-one 
group I (91%) and 19 group II respondents (95%) found the content coverage to be appropriate, 
and, in their view, the curriculum did not warrant major changes. However, both groups 
recommended the inclusion of more examples (65%) and SAQs/TQs /unsolved exercises (74%), 
and that in-between mathematical steps should be worked out in detail to make materials more 
leamer-lfiendly (94%). This validated learner feedback on different courses. Academic 
counsellors would like students to be trained in problem-solving skills that enable them to 
correlate their learning to their lives/surroundings. They also suggested that question banks 
should be developed for each course, and questions of different types, such as multiple choice 
questions in the objective type, numerical in the short answer type, and applications in the long 
answer type, should be included to help learners acquire mastery over the subject. 

A significant majority of respondents in both groups (86%) found the format and style of 
presentation adequate and unanimously recommended that these be retained as such. However, 
their opinion on the need to give more explanations of concepts that were difficult to visualise or 
comprehend was sharply divided; although, the majority (60%) viewed them to be adequate. 
About 76% of responding counsellors consulted IGNOU materials while preparing their own 
classroom lectures. It was encouraging to discover that 84% of responding counsellors would like 
to procure these materials for their library so that their full-time students may use them. This is an 
indication of the national acceptability of the quality of the physics materials, and it may be 
regarded as a spin-off effect of IGNOU’ s efforts to meet a felt need for good text materials at an 
affordable cost. This should help bridge the gap in the quality of higher education provided by 
different universities and colleges in India. 

Eighty-three percent of learners reported that owing to higher standards, a self-leamer needed 
outside help. This perception was confirmed by the majority of counsellors. When a question was 
raised about the books/reference materials consulted by the learners, it was astonishing to learn 
from the counsellors’ feedback that a significant number of learners looked for ‘physics made 
easy’ types of guidebooks. This has implications for the course developers to produce more 
learner-friendly study materials. 

The majority of group I respondents (61.8%) rated the materials as very good. They found the 
language lucid and its comprehensibility high. As such, most of the figures were considered good; 
however, in their view, coloured or 3-D diagrams would improve quality further. Some very 
useful suggestions on deletion/reorganisation/inclusion of content were also made. 

Separate tools were administered on programme designers/developers/experts. The programme 
designers and developers, including the in-house faculty, are subject experts, and they guide the 
process of material design/development by maintaining a fine balance between established and 
emerging bodies of knowledge. As a result, it is expected that learners are trained in the latest 
trends and practices in addition to the essentials. In these investigations, the researchers solicited 
responses from 35 experts. 
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Their feedback revealed that 

• the undergraduate physics curriculum was relevant, up-to-date, and at par with the best 
universities in the country; 

• courses had been pitched appropriately but due to limitations on the number of pages in 
printed blocks, the content density might appear high; 

• learners should be provided separate booklets to put greater emphasis on problem 
solving; and 

• IGNOU should start a M.Sc. in Physics programme with options for applied as well as 
theoretical fields. 

It may be pointed out here that IGNOU’s Physics faculty has undertaken detailed revision of 
learning materials on the basis of the feedback obtained from different stakeholders. In this 
process, the course contents have also been revised with additions and deletions, where 
appropriate, to avoid repetitions and to update the curriculum. The revised study materials for the 
first- level courses are to be sent to learners for the January 2010 session, and their impact on 
learner performance shall be assessed in due course of time. 

Conclusions and Recommendations 

The learners in the sampled population were predominantly urban, adult males, who lived far 
from the study centres. Their seriousness about studying the courses is beyond doubt, though they 
need to be guided clearly about how best to study the courses. Since enrolment in the programme 
grew with time, one can say that the University has succeeded in providing access to higher 
science education to a diverse learner population across the country. However, a high attrition 
rate (73%) and a low success rate have significant implications for the students, the programme 
developers, and the University. Though course materials have been rated to be of high quality, 
relevant, useful, and on par with the best in the country, the need to improve the presentation of 
the courses and to simplify the mathematical details is also emphasised by all concerned. 

The authors recommend that course developers include more and varied in-text questions. Also, 
the problems should correlate learning with real-life situations, and the questions should be 
sequenced in order of difficulty and of learning objectives. The level of available materials must 
be moderated, and important definitions should be reiterated and highlighted in separate text 
boxes. As well, more illustrations, tables, and solved and unsolved examples are needed, and all 
in-between mathematical steps should be worked out. Finally, question ha nk s with answers 
should be developed and provided to learners to facilitate their preparation for term-end 
examinations and to improve their satisfaction levels. 


121 


Evaluation of the Undergraduate Physics Programme at Indira Gandhi National Open University: A Case Study 

Mishra, Vijayshri, and Garg 


References 

Calder, J., & Patel, I. (2002). A monitoring and evaluation framework for technology-based 

community learning centres in India. In H.P. Dikshit, S. Garg, S. Panda, & Vijayshri 
(Eds.), Access and equity: Challenges for open and distance learning (pp. 214-224). New 
Delhi: Kogan Page. 

Fozdar, B.I., Kumar, L.K., & Kannan, S. (2006). A survey of study on the reasons responsible 

for student dropout from the Bachelor of Science programme at Indira Gandhi National 
Open University. The International Review of Research in Open and Distance Learning, 
7{-i). 

Pradhan, B., & Ramanujam, P.R. (2006). Handbook 12: Programme evaluation in open and 
distance education. New Delhi: STRIDE, IGNOU. 

Kapoor, B. (2004). Development of an evaluation model for nursing programme through 

distance education (Unpublished doctoral thesis). Indira Gandhi National Open 
University, New Delhi, India. 

Kellaghan, T., & Stufflebeam, D. E. (2003). International handbook of educational evaluation 
(Vol. I). Boston: Kluwer Academic Publishers. 

Khare, P., Garg, S., & Saxena, A. (2004). Performance analysis of IGNOU science students in 
theory courses through data mining. Indian Journal of Open Learning, 13, 29-49. 

Mishra, A. (2008). Evaluation of undergraduate physics programme of IGNOU and implications 
for learners (Unpublished doctoral thesis). Indira Gandhi National Open University, New 
Delhi, India. 

Mishra, A., Vijayshri, & Garg, S. (2008, in press), A preliminary evaluation of undergraduate 
physics laboratory instruction offered at IGNOU. AAOU Journal of Open Learning. 

Panda, S. (1991). Programme evaluation in distance education: A perspective and proposed 
agenda of actions. Educational technology. Vol. 1 (pp. 168-181). New Delhi: AIAET. 

Quimbo, Maria Ana T. (2002). A model for evaluating distance education programme. In H.P. 

Dikshit, S. Garg, S., Panda, & Vijayshri (Eds.), Access & equity: Challenges for open 
and distance learning (pp. 1 96-2 1 3). New Delhi: Kogan Page. 

Stufflebeam, D. E. (1983). The CIPP model for program evaluation. In G.F. Madaus, M. Scriven, 
& D.E. Stufflebeam (Eds.), Evaluation models (pp. 117-141). Boston: Kluwer-Nijhoff 

Vijayshri, Garg, S., & Panda, S. (1998). Teaching science at the Indira Gandhi National Open 


122 


Evaluation of the Undergraduate Physics Programme at Indira Gandhi National Open University: A Case Study 

Mishra, Vijayshri, and Garg 


University. In S. Garg, S. Panda, & Vijayshri (Eds.), Science and technology through 
open and distance education (pp. 109-172). New Delhi: Aravali International. 


I © I 


S0WERI6HTS RESECTED I 



Athabasca 

University 


123 


